Key Indicators and Differential Monitoring

NARA's approach to Key Indicators and Differential Monitoring is grounded in regulatory science, focusing squarely on Fiene's theory of regulatory compliance which clearly demonstrates the efficacy of the risk assessment and key indicator methodologies as being the most cost effective and efficient ways to implement differential monitoring. Click HERE to learn more.

  • Differential monitoring is a method used to determine the frequency of inspections needed and the scope that will be required based on the setting’s licensing history. This method focuses measuring tools to determine the number and scope of inspections that should be conducted at a licensed setting.
  • Integrated Monitoring is another approach to program monitoring which integrates regulatory compliance and quality into a continuous approach. Quality standards are the focal point of integrated monitoring systems.

Focused measuring tools are licensing inspection instruments that increase the effectiveness and efficiency of a regulatory oversight agency without producing recurring operational costs. In other words, focused measurement tools maximize performance while minimizing costs.
Click 
HERE to learn more.

  1. Weighted Risk Assessment assigns weighted scores to licensing regulations based on the level of risk to persons in care in the event of regulatory noncompliance. Click HERE to learn more.

  2. Key Indicator Systems (KIS) identify a subset of licensing regulations from an existing set of regulations that statistically predict regulatory compliance with the entire set of regulations. Click HERE to learn more.

  3. Quality Indicator Systems identify a subset of licensing regulations and other non-regulatory factors to predict the overall quality of care provided in a licensed setting. Click HERE to learn more.

The purpose of a licensing indicator system is to increase the efficiency and effectiveness of an existing licensing system by refocusing the emphasis of the licensing process. It is intended to complement, and not replace, an existing licensing measurement system.

Additional information, including the Schedule of Services for Measurement Tools, can be viewed on NARA – The Right Choice. 

 


Resources

Studies and Projects

Foundational Studies

  • Colorado, 2014 - This study analyzed Colorado's quality rating system, the Qualistar Rating, for generating key indicators. Read the full study.
  • Georgia, 2014 - Validation of Georgia's process for determining if state-regulated child care facilities are compliant with basic state health and safety requirements. Read the full study.
  • Oregon, 2013 - This study provided Oregon's OCC with a basic risk factor analysis comparing its child care center rules to Stepping Stones (SS) standards. Read the full study.
  • Head Start, 2013  - This project provided the Office of Head Start (OHS) Key Indicators for their Head Start Performance Standards. Read the full study.

Differential Monitoring Projects (NARA) 

  • Saskatchewan 2022 - 2023
  • Focus: The Saskatchewan Ministry of Education has completed the design, development, and validation of a new Program Quality Indicator (PQI) Tool/ScaleSaskatchewan Program Quality Indicators Tool This is the last component to their design of a comprehensive differential monitoring system.
    Project Activities: Key indicators methodology, Quality indicator development, regulatory analysis assessed with program quality, demonstration of the regulatory compliance ceiling effect, use of the regulatory compliance scale, data collection and data analysis for the development of the quality key indicators, center based infant, toddler and preschool classrooms, use of the ERS scales for validation purpose, validation and reliability testing of scale, independent observers/data collectors utilizing PQI and ERS scales, and development of the PQI scale. Read the Full Report here. 
  • Florida, 2021
    Focus: Development of a Differential Monitoring Key Indicator System (KIS) for the Florida Department of Children of Families, Office of Child Care (OCC).
    Project Activities: stakeholder feedback sessions, data collection and analysis, development of an inspection checklist, policy and procedure development, and staff and stakeholder training. Read the Full Report here.
  • Alberta, Canada, 2020 - 2021
    Focus: Development of Alberta Child Care’s licensing Key Indicator System (KIS) as well as additional analyses and recommendations based on the findings for Alberta Child Care’s Child Care Centers, Preschool Programs, and Out of School Care (OSC).
    Project Activities: data collection and analysis to determine key indicators. Read the Licensing Key Indicator Predictor Rule Results Report here - coming soon.
  • Washington, 2019 - 2020
    Focus: Validation of Washington State’s Child Care Risk Assessment and Licensing Decision Making Tiered System
    Project Activities: validation study of Washington State’s Department of Children, Youth and Families child care Risk Assessment Licensing Decision Making Tiers System (RALDMTS). The validation involves two key components: 1) Validation of the measurement strategy used to determine the licensing decision making for child care centers and family child care homes; 2) Validation of the licensing system in juxtaposition to the program quality measures (ERS & CLASS) as part of their QRIS – Quality Rating and Improvement System utilized in Washington. Read the full report here.
  • Washington, 2019 - 2020
    Focus: Licensing Measures and Outputs Validation Study for the Department of Children, Youth, and Families Child Care
    Project Activities: Validation of the measurement strategy used to determine the licensing decision making for child care centers and family child care homes; Validation of the licensing system in juxtaposition to the program quality measures (ERS & CLASS) as part of their QRIS – Quality Rating and Improvement System. Read the full report.
  • Saskatchewan, Canada, 2019 - 2021
    Focus: Key Indicator study and Weighted Risk Assessment study for the Ministry of Education in the Province of Saskatchewan
    Project Activities: Key Indicator study as well as trends in regulatory compliance; Home-based Weighted Risk Assessment.  Read the full Key Indicator reportRead the full Weighted Risk Assessment reportRead the full CCC/FCC Validation Study Report here.Read the Full Report here.
  • Minnesota, 2019
    Focus: Family Child Care Key Indicator Study for Minnesota Department of Human Services Licensing Division
    Project Activities: Key Indicator Data Analysis. Read the full report. Follow-up: Minnesota Family Child Care Task Force - Interim Legislative Report, March 2020
  • Indiana, 2018 - 2019
    Focus: Key Indicators for the Family and Social Services Administration, Office of Early Childhood and Out of School Learning
    Project Activities: Focus Groups; Survey & Data Collection; Statistical Analysis; Indicator Work Tools & Manual; Training. Read the full report.
  • Montana, 2016 - 2017
    Focus: Development of Key Indicators for Department of Public Health and Human Services
    Project Activities: Statistical Analysis; Policy Development; Staff Training; Stakeholder Meetings. Read the full report.
  • Michigan, 2016 - 2017
    Focus: Development of Key Indicators for licensed child care and group homes and registered family homes
    Project Activities: Data Collection & Analysis; Policy Development; Work Tools and Instruction Manual; Staff Training; Stakeholder Meetings; Pilot Program; Statewide Implementation. Read the full report.
  • Ontario, 2015 - This project outlined recommended methodology for developing an abbreviated inspection tool; a tiered licensing approach to child care licensing. Read the full report.
  • Illinois, 2013 - 2014
    Focus: Development of a Weighted Licensing and Key Indicator System
    Project Activities: Determination of Eligibility Criteria; Data Collection & Analysis; Identification of Additional Factors; Development of Indicator Work Tools & Instructional Manual; Staff Training; Stakeholder Meetings. Read the full report. Follow-up: Illinois DCFS - Day Care Licensing Annual Report to the General Assembly, FY 2019
  • Kansas, 2012 - 2013
    Focus: Develop a Key Indicator System
    Project Activities: Data Capacity Building and Analysis; Policy Development; Stakeholder Meetings. Read the full report.
  • Washington, 2011 - This project provided Washington's Department of Early Learning a differential monitoring process to ensure efficiency while keeping children in licensed safe care. Read the full study.